Optimal Cross-Validation Split Ratio: Experimental Investigation

Cross-validation is a widespread method for assessing the generalisation ability of a model in order to tune a regularisation parameter or other hyper-parameters of a learning process. The use of cross-validation requires to set yet an additional parameter, the split ratio. Few texts have investigated theoretically the asymptotic setting of this ratio, and no consensus has emerged. In this contribution, we investigate the sensitivity and optimal setting of the split ratio on a particular model, a non-parametric kernel estimator with adaptive metric. 1 Cross-validation Most eecient learning procedures require the setting of an extra learning parameter , or \hyper-parameter". Neural networks typically use a regularisation parameter weighting a weight decay 1], or the extent of pruning 2]. Estimating the \optimal" hyper-parameter is the topic of active current research in the statistical learning community 3]. Let us consider a typical learning problem: modelling an input-output relationship based on some empirical data